Efficient Image Captioning Based on Vision Transformer Models

Image captioning is an emerging field in machine learning. It refers to the ability automatically generate a syntactically and semantically meaningful sentence that describes content of image. requires complex learning process as it involves two sub models: vision sub-model for extracting object features language use extracted captions. Attention-based transformers models have great impact recently. In this paper, we studied effect using on image by evaluating four different transformer sub-models The first used DINO (self-distillation with no labels). second PVT (Pyramid Vision Transformer) which not convolutional layers. third XCIT (cross-Covariance changes operation self-attention focusing feature dimension instead token dimensions. last one SWIN (Shifted windows), which, unlike other transformers, uses shifted-window splitting For deeper evaluation, mentioned been tested their versions configuration, evaluate model five backbones, versions: PVT_v1and PVT_v2, XCIT, transformer. results show high effectiveness SWIN-transformer within proposed regard models.